June 15, 2017

Introduction/Purpose

This set of slides aim at providing some guidelines for reporting statistical analyses and methods.

Loosely based on the SAMPL Guidelines for Biomedical Journals by Thomas Lang and Douglas Altman (Lang and Altman 2015).

THE General Principles

From Lang and Altman (2015):

Number 1:

Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to verfy the reported results.

  • I.e., make sure your analysis is reproducible for someone with the right statistical knowledge, but no prior knowledge about your work.

Number 2:

Provide enough detail that the results can be incorporated into other analyses.

The next section is divided into two parts: the first deals with how to report the details of the statistical methods used, the second with how to report the results.

Reporting Statistical Methods

Preliminary analyses

  • Study design
    • Describe the study design as accurately as possible
    • Sample size/power calculations (if you're not convinced power calculations are necessary: check this out)
    • Inclusion/Exclusion criteria: if not done proberly, excluding subjects could bias results
  • Variable transformations, new variable creation, …
    • log transformed to achieve normality, mean of variables used as explanatory variable, continuous data collapsed into categories, etc.
    • not a bad idea to mention if NO transformations where used (depending on methods used)

Reporting Statistical Methods

Primary analyses

  • Describe the purpose upfront by stating your hypothesis clearly.
    • This makes p-hacking less likely to occur (more on that later)
  • Describe the method used to test your hypothesis in full
    • Again, any transformations used, variables included
  • Include descriptive statistics
    • means and standard deviations
    • if data is not normal, median and range/IQR (Inter Quartile Range)
  • Justify your choice of method
    • Was the model validated by checking the assumptions?
      • Check for normality; if data not normal, use non-parametric test (Wilcoxon instead of t-test, Kuskal-Wallis instead of ANOVA, …)
      • Linear regression: make sure to check for normality, independence, constant variance, linearity
  • Multiple comparisons: Adjust p-values for multiple comparisons (see more here, slide 17-18)
  • Always, always, ALWAYS specify the program used in the analysis.

Reporting Statistical Results

Numbers and Descriptive Statistics

References

Lang, Thomas A., and Douglas G. Altman. 2015. “Basic Statistical Reporting for Articles Published in Biomedical Journals: The ’Statistical Analyses and Methods in the Published Literature’ or the Sampl Guidelines.” International Journal of Nursing Studies 52 (1): 5–9. doi:http://dx.doi.org/10.1016/j.ijnurstu.2014.09.006.